Reasoning Constructively about Probability and Programming with Chance

Probability and statistics are the deductive and inductive theories, respectively, of chance and uncertainty. However, they typically do not have a role in traditional logic and theorem proving. This is unfortunate, because chance is an important concept in computer science. The interaction between computer science and uncertainty typically takes three forms. First, algorithms may make random decisions and find solutions to difficult problems quickly with high probability. For example, GSAT is a random SAT solver which typically beats deterministic solvers such as the Davis-Putnam procedure. AI techniques such as genetic algorithms and simulated annealing are also randomized. Some randomized algorithms suggest efficient exact or approximate deterministic algorithms. Randomized algorithms rely on probability theory to reason about the consequences of generating data which is unpredictable and so (with high probability) does not contain pathological “worst cases” which slow algorithms down. Second, many techniques in artificial intelligence use probabilistic reasoning about knowledge to make decisions. Such techniques include Bayesian networks, probabilistic parsing in natural language processing, and logics of probability, knowledge and belief. These approaches rely on probability theory to explain how to calculate the probabilities of various possibilities so that good decisions can be made. Finally, many arguments about the optimality or average case behavior of deterministic algorithms rely on probabilistic reasoning. Some examples include average-case analyses of sorting algorithms like quicksort and of list-reordering heuristics like move-to-front, and reasoning about the optimality of compression algorithms like Huffman coding. This approach relies on the theory of probability for an understanding of the average cases of algorithms or the typical sequences generated by data sources with given probability distributions. Thus, there is clearly a place for chance in systems for reasoning about computation such as Nuprl. However, it is not clear which of the many alternative known approaches to human or automatic reasoning about chance is most suitable for expression in constructive type theory. In this report I will describe some of the main approaches taken by mathematicians, logicians, and computer scientists in formalizing uncertainty. I will outline the advantages and disadvantages of formalizing each approach in constructive type theory. I will also propose a simple new approach based on some of these ideas which I believe allows both effective programming and effective reasoning with probability and randomness. For simplicity, I will only consider finite cases of these theories, since infinite objects are difficult to deal with computationally. I have grouped the approaches I have investigated into three broad categories. The first category consists of the standard set-theory and measure-theory based mathematical theories of probability. The second set of approaches are theories relating logic and standard probability in a computational framework. The third group of approaches comprises alternative theories of probability, uncertainty, or randomness, which may be better places to start formalizing probability than the standard theories.